1. Field of the Invention
The present invention relates to a motion vector detector used in coding of image and reduction of noise, and a video coder for transmitting and accumulating images in a small coding quantity.
2. Related Art of the Invention
A conventional technique for detection of motion vector is described, for example, by K. FUKINUKE in "Image digital signal processing" (chapter 10, NIKKAN KOGYO SHIMBUN, 1985). Herein, the "method of minimizing the difference between consecutive images" is to determine the motion vector as the shift (a, b) for minimizing formula 1, where g.sub.t (x,y) denotes the luminance at screen position (x,y) in frame t, and R represents a block region on the screen. ##EQU1##
This principle is shown in FIG. 9. The motion vector is determined as the shift (a,b) to block R' having the closest correlation with block R, in the image of frame of time t-1. In formula 1, the sum of absolute values is determined as the evaluation of correlation, but the sum of squared errors may be also used. Such "method for minimizing the difference of consecutive images" is herein called the block correlation method.
The block may not be necessarily square. For detection of motion vector by block correlation method, the following problems exist.
(A1) When the block region is set smaller, the reliability of detected motion vector is lowered due to disturbance of the noise component. To the contrary, when the region is larger, the detection precision is higher, but the granularity of motion vector detection becomes coarse.
(A2) If there is no luminance change component differing in direction in the region defined by the block, the correspondence of region on the frame is not determined.
To solve these problems, several improvements of block correlation method had been attempted. For example, as prior art for solving the problem (A1), a motion vector detecting method (prior art 1) was disclosed in the Japanese Laid-Open Patent No. 62-230180. This is a method of setting wider so that mutual blocks may overlap, instead of equally dividing the image to form blocks as the basic unit corresponding to a region between frames. According to this method, if the interval of motion vector detection is narrow, the block for calculating the correlation is greater than the interval of motion vector detection, and therefore the motion vector may be stably estimated from the block correlation equally divided by the interval of motion vector detection.
As prior art for solving the problem (A2), for example, a motion vector detector (prior art 2) was disclosed in the Japanese Laid-Open Patent 62-105587. In this apparatus, by determining the variance of luminance values in the block, it is evaluated whether the block for calculating the correlation is valid for motion vector detection or not. If the variance of luminance in the block is small, the use of a motion vector low in reliability can be avoided as an invalid vector not determined in the correspondence of a region on different frames. Two prior arts of motion vector detecting method or apparatus have been introduced.
Next is shown prior art of video coder. For example, a video coder (prior art 3) is mentioned in CCITT Recommendation H.261. In the inter-frame coding mode of this coder, when coding the present frame image, a predicted image of the current frame is produced as a motion compensated image from the image of the preceding frame by the block correlation method, and the differential image between the motion compensated image and current frame image is coded. In this coder, while the motion compensated image is matched with the preceding frame without error, the information to be sent is only the motion vector, so that the image can be transmitted by a small coding amount.
Incidentally, the H.261 is a specification of a video coder recommended for the purpose of transmitting an image of at least 144.times.180 pixels at a coding rate of 64 kilobits per second or faster. If an image of similar size is coded at about 20 kilobits per second (hereinafter called very low bit rate coding), the following problems are experienced.
(B1) A motion compensated image is created by using parallel move in a block unit determined by the detected motion vector. Generally, hence, the motion compensated image is discontinuous on the block border. If the coding amount that can be transmitted is enough, this block border is not recognized as it is transmitted as the differential information between the motion compensated image and current frame image. If the coding amount is limited, however, it is recognized as a visual disturbance.
(B2) In the video coder conforming to the H.261, high efficiency coding is realized by a discrete cosine transform of the image in every block and coarse quantization of high frequency components. In a limited coding amount, however, a quantizing error of a DC component in each block is recognized as the block border and becomes a visual disturbance.
As prior art for solving the problem (B1), for example, an image coding method (prior art 4) was proposed by M. KAWASHINA et al. in "Ultralow bit rate coding of motion picture" (Technical Report of Institute of Electronic Information and Communications Engineers of Japan, EI92-117, February 1993). This is a method for creating a motion compensated image by determining the motion amount of each pixel by interpolation of the transmitted motion vector. According to this method, a smooth motion compensated image is obtained by a small number of motion vectors.
As a prior art for solving the problem (B2), for example, a sub-band coding method (prior art 5) was disclosed in the Japanese Laid-Open Patent No. 62-230180. In the sub-band coding method, the picture is divided into different frequency bands by filter scanning. In the former prior arts, the sub-banded image, or the image composed of different frequency bands can be coded with high efficiency, by scanning the sample point at the same spatial position from the low frequency to the high frequency region. In this sub-band coding method, by contrast, since luminance information produced from the frequency components of the image are overlapped, the quantizing error of the low frequency components is not recognized as the block border to cause a visual disturbance.
In spite of the prior art, however, the following problems exist.
(C1) In prior art 1 and prior art 2, the motion vector is not determined by making use of a correlation of adjacent blocks. In prior art 1, it is possible that a motion vector largely different from an adjoining block may be detected. In prior art 2, if desired to determine the motion vector of an invalid block, there is no other method than to determine by interpolation of the motion vector obtained in the adjoining block.
(C2) In very low bit rate coding, it is necessary to decrease not only the coding amount of differential image, but also the coding amount of motion vector. In the video coder conforming to the H.261 as prior art 3, the coding amount is kept low by Hafman coding of the difference from the adjoining motion vector. However, the correlation of motion vectors used in coding is in one dimension and one direction only, and a more efficient motion vector coding is demanded in very low bit rate coding.
(C3) In prior art 4, in the first place, the motion vector is determined by block correlation method, and the motion vector in pixel unit is determined by a luminance gradient, then the representative motion vector is corrected and determined so that the differential signal power between frames may be a minimum. In this prior art, the motion vector in a pixel unit can be determined by interpolating few representative motion vectors, but the disclosed technique is complicated.
(C4) Even by employing the coding method of prior art 5, if attempted to perform very low bit rate coding by sub-band division, it is necessary to quantize the sub-band divided images coarsely. As a result of decoding by synthesizing coarsely quantized frequency components, ringing is recognized as a visual disturbance.